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DECISION ON APPEAL 

This is an appeal from the final rejection of claims 1 through 5, 7 through 15, 17 through 
24, 26 through 33 and 35 through 38. 1 

The disclosed invention relates to a method and system for creating visual summaries of 
video material. The method and system receive keyframes of video material, and extract frame 
signatures from the keyframes. Superhistograms are then created from the frame signatures. The 
frame signatures and the superhistograms are thereafter used to select representative keyframe 

1 Claims 17 and 35 erroneously depend from canceled claims 16 and 34, respectively. 



Appeal No. 2006-1591 
Application No. 09/866,394 



images for each superhistogram to create a compact visual summary of the video material. 
According to the disclosed and claimed invention, the representative images include at least one 
of (1) the first image in each family histogram, (2) the most meaningful image in each 
superhistogram, (3) a randomly chosen image, and (4) an image that is closest to the cluster 
center. 

Claim 1 is illustrative of the claimed invention, and it reads as follows: 

1. For use in a system (100) capable of creating visual summaries of video material, an 
apparatus (130, 200) for creating a compact visual summary of video material, said apparatus 
(130, 200) comprising: 

a visual summary controller (130, 200) capable of receiving keyframes of said video 
material; 

wherein said visual summary controller (130, 200) is capable of extracting frame 
signatures from said keyframes, and capable of using said frame signatures to create 
superhistograms from said keyframes, and capable of using said frame signatures and said 
superhistograms to select representative keyframe images for each superhistogram to create a 
compact visual summary of said video material, 

wherein said representative images include at least one of (1) the first image in each 
family histogram, (2) the most meaningful image in each superhistogram, (3) a randomly chosen 
image, and (4) an image that is closest to the cluster center. 

The references relied on by the examiner are: 

Wang et al. (Wang) 5,805,733 Sept. 8, 1998 

Dimitrova et al. (Dimitrova), "Color SuperHistograms for Video Representation," Proceedings of 
the IEEE International Conference on Image Processing , October 1 999, pages 3 14 through 318. 
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Claims 1 through 5, 7 through 15, 17 through 24, 26 through 33 and 35 through 38 stand 
rejected under 35 U.S.C. § 103(a) as being unpatentable over Dimitrova in view of Wang. 

Reference is made to the final rejection, the briefs and the answer for the respective 
positions of the appellants and the examiner. 

OPINION 

We have carefully considered the entire record before us, and we will sustain the 
obviousness rejection of claims 1 through 5, 7 through 15, 17 through 24, 26 through 33 and 35 
through 38. 

According to the examiner's findings (final rejection, pages 2 and 3), the Dimitrova 
publication discloses all of the steps and structure of claims 1, 11,21 and 30 with the exception 
of selecting "representative keyframe images for each superhistogram to create a compact visual 
summary of said video material, wherein said representative images include at least one of (1) the 
first image in each family histogram, (2) the most meaningful image in each superhistogram, (3) 
a randomly chosen image, and (4) an image that is closest to the cluster center." The teachings of 
Wang were relied on by the examiner because they describe summarizing video sequences to 
create compact visual summaries of video material by "selecting representative keyframe images 
from each group of related scenes to create a compact visual summary of the video material 
(summarizing a video sequence by taking one representative frame from each set of related 
scenes with similar average color histograms, to represent the set to enable the user to view a 
large sampling of video sequence images) (Wang et al.: column 1, lines 51-67 and column 2, 
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large sampling of video sequence images) (Wang et al.: column 1, lines 51-67 and column 2, 
lines 1-24; this is further shown in Figure 3), wherein the representative images include at least 
one of the first image in each family histogram, the most meaningful image in each 
superhistogram, a randomly chosen image and an image that is closest to the cluster center (the 
representative frame image can be taken from the temporally medial scene in the set or from one 
of the frames of the longest scene in the set of related scenes) (Wang et al.: column 3, lines 37- 
66)" (final rejection, page 3). In view of the teachings of Dimitrova and Wang, the examiner 
concluded (final rejection, pages 3 and 4) "[i]t would have been obvious to one of ordinary skill 
in the art ... to modify the visual summary controller capable of extracting frame signatures 
from keyframes to create superhistograms of Dimitrova et al., to include the further step of 
selecting representative keyframes from those superhistograms and using the representative 
keyframe images to create a compact visual summary, taught by Wang et al." 

Appellants agree with the examiner's findings concerning the teachings of Dimitrova 
(brief, page 9). On the other hand, appellants disagree with the examiner's findings concerning 
Wang because appellants are of the opinion that Wang describes temporal ordering of frames 
whereas the disclosed invention is based on non-temporal ordering of frames (brief, pages 11,16 
and 18). Based upon this difference, appellants argue that Wang fails to describe representative 
images that include at least one of "the most meaningful image in each superhistogram" or "an 
image that is closest to the cluster center." 
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In reply to the appellants 5 temporal versus non-temporal argument (answer, page 8), the 

examiner correctly noted "there is no basis or support in the specification or claim language for 

the applicant's [sic, applicants 5 ] argument. 5 ' Thus, we agree with the examiner's observation 

(answer, page 10): 

Since the specification of the applicant's [sic, applicants'] 
invention fails to include a definition of "cluster center", taking the 
broadest reasonable interpretation, one of ordinary skill in the art 
would interpret an image that is closest to the cluster center to be 
an image that is closest to the center, or middle of a cluster of 
scenes. Since Wang teaches that the representative frame is a 
frame that is halfway between the first and last scenes, Wang 
teaches that the representative frame is at the center of the cluster 
of frames and is therefore a frame that is closest to the cluster 
center. 

With respect to appellants' argument concerning "the most meaningful image," we agree with the 

examiner's statements (answer, pages 10 and 11): 

Second with reference to the representative image being the most 
meaningful image in each superhistogram, Wang teaches that the 
representative frame can be a frame taken from the longest scene, 
since the longest scene is most indicative of the content of the 
related scenes, as recited in column 3, lines 59-62. The applicant 
argues [sic, applicants argue] that simply selecting one frame from 
among the frames of the longest scene, as taught by Wang, does 
not teach or disclose the "most meaningful frame in the group" 
since the specification of the invention recites with particularity (by 
example) what constitutes the "most meaningful frame", i.e. a 
person's face, an important text, etc. The examiner respectfully 
disagrees. In response to applicant's [sic, applicants'] argument 
that the references fail to show certain features of applicant's [sic, 
applicants'] invention, it is noted that the features upon which 
applicant relies [sic, applicants rely] (i.e., the representative frame 
being a person's face, an important text, etc.) are not recited in the 
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rejected claim(s) .... Although the applicant's [sic, applicants 5 ] 
specification gives examples of some frames that can be 
considered the "most meaningful frame" (the specification recites 
at page 16, lines 21-22, "The term 'meaningful image 5 may refer to 
a frame with a person's face, an important text, etc.' 5 ), it does not 
require that the most meaningful frame has to be a frame with a 
person's face or an important text. Wang teaches selecting 
representative scenes for presentation to a user that have the "most 
significant content", as recited in column 5, lines 1 1-20. Wang 
further recites that a representative frame "can be taken as one of 
the frames of the longest scene in a set, the longest scene being 
most indicative of the content of the related scenes 5 ', in column 3, 
lines 59-62. Therefore, the examiner respectfully argues that since 
the longest scene is most indicative of the content of the related 
scenes, the longest scene is the most meaningful of the group. 

We agree with the examiner's conclusions that Wang describes representative images that 
are "closest to the cluster center," and a representative image that is "the most meaningful 
image." It appears that appellants also agree with the examiner's findings (reply brief, page 2, 
paragraphs 4 through 6). 

In summary, the obviousness rejection of claims 1, 11,21 and 30 is sustained. The 
obviousness rejection of claims 2 through 5, 7 through 10, 12 through 15, 17 through 20, 22 
through 24, 26 through 29, 31 through 33 and 35 through 38 is likewise sustained because 
appellants have not presented any patentability arguments for these claims apart from the 
patentability arguments presented for claims 1, 11,21 and 30. 

DECISION 

The decision of the examiner rejecting claims 1 through 5, 7 through 15, 17 through 24, 
26 through 33 and 35 through 38 under 35 U.S.C. § 103(a) is affirmed. 
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No time period for taking any subsequent action in connection with this appeal may be 
extended under 37 CFR § 1.136(a)(l)(iv). 

AFFIRMED 




KENNETH W. 
Administrative Patent Judge 




ERROL A. KRASS 
Administrative Patent Judge 

HOWARD B. BLANKENSHIP ^ 
Administrative Patent Judge 
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